Accepted May 11, 2021; Epub ahead of print May 26, 2021

J Rehabil Med 2021; 53: jrm00213

Correspondence address: Jan Johansson, Department of Clinical Neuroscience, Eye and Vision, Marianne Bernadotte Centre, Karolinska Institute, St Erik Eye Hospital, 112 81 Stockholm, Sweden. E-mail jan.johansson.1@ki.se

Doi: 10.2340/16501977-2847

Some survivors of profound acquired brain injury develop a prolonged disorder of consciousness. Differentiation of the stage of prolonged disorders of consciousness is of relevance when planning rehabilitation and informing families. It currently relies largely on clinical assessment using standardized neurobehavioural rating scales. In the Coma Recovery Scale Revised (CRS-R) the patient is encouraged to perform tasks while the clinician observes. The patient’s responses may, however, be difficult to interpret due to impairments associated with the brain injury. The objective of this study was to develop eye-tracking as a supporting tool to CRS-R assessment of patients with prolonged disorders of consciousness. A selection of subtests in the CRS-R were performed while recording eye movements with a wearable eye-tracker. No major adverse reactions were observed, suggesting likely patient acceptability. Distinct eye movements were discernible from the recorded data, and analysis of these gave results with the potential to support clinical assessment. Further study is merited.

Following profound acquired brain injury in adults, a small group of patients begin to open their eyes with apparent sleep-wake cycles, but show no or extremely limited behavioural signs of consciousness. When this state is due to brain injury and persists for longer than 4 weeks it is described as a prolonged disorder of consciousness (PDOC) (1).

Differential diagnosis of PDOC is an important element in assessing prognosis and in subsequent clinical decision-making. Specifically, the differentiation between unresponsive wakefulness syndrome (UWS, also called vegetative state) (2) and minimally conscious state (MCS) (3) is of relevance when planning rehabilitation interventions and informing families; MCS has a more favourable prognosis. Reproducible visual fixation and visual pursuit may be early indications that the patient is in MCS rather than UWS.

After exclusion or minimization of other factors that can impact behavioural responses (e.g. sedation due to medications, critical illness neuro/myopathy, potential pituitary dysfunction and other medical complications), and clinical examination, use of standardized behavioural assessment scales is necessary to establish diagnosis (4–5).

The Coma Recovery Scale Revised (CRS-R) (6) is the most established standardized scale, for example being recommended by the American Academy of Neurology (7–9). In the CRS-R the patient is asked to perform simple tasks involving basic levels of conscious responses while the clinician observes and gives ratings, with the aim of distinguishing stimuli-related responses (considered to be conscious responses) from randomly occurring or reflexive responses. Observations of eye movements play an important role in the assessment.

The patient’s responses during the CRS-R may be hampered by several factors such as cognitive, linguistic, physical and motor impairments associated with the brain injury as well as level of arousal and fluctuation of functioning (10). Even if the patient’s intention is to respond to an instruction, e.g. to look at an object, the response may be poor and difficult to interpret. Strabism (squint), gaze palsies or impaired volitional motor control over eye movements may add further uncertainty to the assessment. Criteria for awarding points on the CRS-R are necessarily fairly demanding, in order to minimize the risk that random occurrences are misinterpreted as intentional. However, examining clinicians may get a sense that the patient “seems to be trying” to perform the task, even though performance falls short of the scale criteria. It is currently difficult to know what, if any, emphasis should be placed on such clinical impressions in forming a diagnostic assessment. It is therefore of interest to develop supportive tools that can be applied in a clinical setting, with the potential to detect subtle non-random stimulus-related responses that would contribute to early detection of behavioural signs of consciousness (11).

Analysis of eye movements using eye-tracking technology has been suggested as a potential support in differential diagnosis (12–15). Several of the commercially available eye-tracking systems use desktop-mounted set-ups that require the subject to be positioned within certain angles of the sensors. The subject must also be able to focus their gaze within a specific spatial area, so that the sensors can capture the eye movements. These requirements cannot be met by patients with PDOC for whom any communicative ability is by definition minimal and fluctuating and where considerable physical limitations due to paresis and disorders of motor tone are usual. Recent technological advances have led to the development of eye-tracking units that can be worn as a pair of spectacles, which has the potential to reduce some of these difficulties. A balance needs to be found between technical requirements, practicality and patient- friendliness. Methods that allow for calibration difficulties are needed, as well as methods for data analysis when the standard commercially available analysis tool cannot be used.

The objective of this study was to evaluate the feasibility of using a wearable eye-tracker when assessing patients with PDOC using the CRS-R focusing on technical challenges.

The study was carried out as an observational methodological case series with descriptive as well as analytical elements. Four patients were recruited from the rehabilitation clinic of a regional rehabilitation unit. The examinations and tests were performed in an outpatient setting in a hospital environment. A physician (specialist in rehabilitation medicine) with experience with patients with PDOC performed initial screening with regards to inclusion and exclusion criteria.

Inclusion criteria were: (i) adult patients aged 18–65 years who had had an acquired brain injury in adulthood; and (ii) a suspected PDOC. A PDOC was suspected when the patient was unable to demonstrate functional object use and there was an absence of functional communication more than 4 weeks post-injury, but not suspected if aphasia and apraxia were clinically considered to be the primary reason for this (e.g. dominant hemisphere stroke). Exclusion criteria were: known blindness or deafness, eye disease or severe eye motility restrictions, medical instability, sedation, medical restrictions on neck movements and if relatives opposed the patient’s participation in the study.

Consent

The study was approved by the Swedish Ethical Review Authority (Dnr: 2018/150-31). Patients with PDOC, by definition, lack the capacity to give informed consent. Patient’s relatives were informed about the study and patients were included only after the relatives signed a form approved by the ethics authority stating that they had no objection.

Eye-tracking equipment

Eye movements were recorded with a Tobii Pro Glasses 2 eye-tracking system (www.tobiipro.com) with a sampling frequency of 50 Hz. This is a wearable eye-tracker designed to be worn like a pair of spectacles. Corrective lenses can be added to compensate for refractive error or if reading aids are needed. The sensors and a scene camera (90° field of view) are built into the frame. The total weight is approximately 45 g. The scene camera is located centrally in the frame slightly above the eyes and projects straight ahead, in order to capture what the subject sees.

In order to track the eye movements as accurately as possible and, importantly, to be able to map the eye movements in relation to a visual stimulus, the eye-tracking system needs to be calibrated. This procedure involves asking the participant to look carefully at a calibration target with both eyes. The eye-tracker used here requires a 2-eyed calibration. Strabismus is common in the PDOC patient group and, in these cases, it is not possible for the patient to view a calibration target with both eyes simultaneously. Patients with PDOC often cannot fixate on visual targets. The consequence of failed calibration is that the standard analysis tool of the eye-tracking system cannot be used. Another technical issue is that the standard analysis tool of the system does not allow mapping of eye movement recordings against moving targets. Thus it was necessary to investigate work-around solutions.

Coma Recovery Scale Revised and general procedure

The test paradigm included 4 items from 2 of the 6 sub-scales of the CRS-R (Table I), administered according to the CRS-R manual. As there are some tasks for which the CRS-R manual gives a degree of choice regarding stimuli, we give further details here. In item 1 (see Table I) the objects consisted of a yellow ball and a brown cup of similar size (diameter approximately 65 mm). In item 2 a hand mirror (diameter 138 mm) was used. The object in item 3 consisted of a yellow ping-pong ball mounted over a mini flashlight. When the flashlight was switched on the ping-pong ball became luminescent. The auditory stimulus in item 4 was a miniature hand-bell.

Prior to testing careful attempts were made to optimize arousal prior to testing by tactile and sound stimulation. In the context of this research study for patients late after injury it was not considered ethical to perform the arousal facilitation protocol of the CRS-R fully as per the manual, as it relies on the application of painful stimuli. During testing the patient remained seated in a comfortable position in their wheelchair, positioned to face a wall of light neutral colour. The patient’s visual acuity was estimated from their latest spectacle prescription, when available. If this was not available retinoscopy was performed to estimate if there was a refractive error or a need for near aids. Corrective lenses were snapped on to the eye-tracker frame, if judged necessary to allow the patient to see the test objects clearly enough. An allowance of approximately +/–1 dioptre was accepted as test objects were large and due to uncertainty in measures. Before putting the eye-tracker frame onto the patient, a pair of standard spectacles was put on, to familiarize the patient with the situation and to watch for any adverse reactions.

Eye movements recording began with a calibration procedure. The patient was asked to look at a calibration target while the eye-tracking system acquired eye position information. If calibration was unsuccessful, which was most commonly due to strabismus, the subsequent recording of eye movements was performed without calibration. During preparations for testing, the examiner observed where in the room the patient seemed to be looking and with which eye. The movements of this eye were then primarily used in the analysis.

The CRS-R items were assessed as per the CRS-R manual by a psychologist trained in CRS-R assessment. First, spontaneous eye movements were observed for 1 min, while also recording the eye movements with the eye-tracker. Then, the 4 test items from the CRS-R, as noted in Table I, were assessed with brief pauses between items, during which the psychologist made notes of their observations and ratings. The psychologist performed the whole CRS-R scale assessment without subsequent eye-tracking afterwards.

Eye movement recordings were analysed stepwise using Tobii Pro Studio (TBS) or Origin 2017 (www.originlab.com). In the first step TBS was used to replay the recording and divide it into events based on the actual test events seen via the scene camera. Within the eye-tracker recording several “events” were identified, with start at the time-point where the patient first received an instruction (items 1 and 3) or was exposed to a stimulus (items 2 and 4) and end when the stimuli stopped (was removed from view (items 1, 3 and 4) or the sound ended (item 2)). In the second step Origin 2017 was used to plot the eye-tracking data for each separate event on a graph for visual inspection of the integrity of data. Missing data was the main issue and occurred due to eye closure or at extreme gaze angles towards the periphery. In the third step the optometrist assessed whether the patient’s eye movements responded to the stimuli as per the instruction given. This was assessed primarily by visual inspection of data with comparison of trends between pre-test data and data collected during exposure to a stimulus.

If calibration was successful the visualization tool and metrics available in TBS were used as basis for judging the response in item 1 (object recognition). The recorded eye fixations were superimposed on a snapshot of the scene camera’s view using an automated mapping function. The mapping was manually checked for accuracy and corrected if obvious mis-mappings had occurred (usually less than 10% of fixations).

The clinical and the eye-tracker protocols were scored by different individuals blinded to each other’s scores.

Four patients were included in the analysis (Table II). Clinical, practical and eye-tracking-related aspects are presented and discussed first. Following this, technical issues for each test item from the CRS-R are presented.

Retinoscopy

It proved challenging to perform a standard static retinoscopy due to difficulties controlling where the patient focused their gaze. Instead, dynamic retinoscopy was performed with the examiner positioning themselves at the same distance from the patient as that at which the test-objects were to be presented. The patient was verbally encouraged by the examiner to look toward the retinoscope lamp.

Practical experiences of using the eye-tracker

The eye-tracking frame, despite its fairly lightweight construction, is somewhat different from a normal pair of spectacles. The electronics add some bulk to the frame and limit the field of view to some extent. Care was taken to choose nose pads that appeared to rest comfortably on the patient’s nose. It was found that a very small piece of soft padding under the nose pads improved comfort and helped prevent the frame from sliding down the nose. However, this added some extra distance between the eye-tracker cameras and the patient’s eyes. Another aspect was the fact that the ear-pieces of the frame are quite straight, long and somewhat thicker than normal spectacle ear-pieces. This proved to be a challenge, due to the neck rests of the patients’ wheelchairs. A certain amount of trial and error was required when adjusting head position to find a balance between patient comfort and preventing the neck rests from pushing the frame out of position. When needed, a neck strap was used to keep the frame in place. In general, the patients appeared to tolerate the eye-tracking frame fairly well. However, in one case it was observed that the patient kept their eyes closed to a greater extent when wearing the eye-tracker. This was confirmed by re-testing without the frame on. The finding may be interpreted as a potential comfort issue. Given these experiences from practical use, some adaptations of the frame design may prove beneficial.

Calibration

Each test-session began with an attempt to calibrate the eye-tracker. An assistant positioned themselves directly in front of the patient, holding the calibration target in the patient’s theoretical line of sight. Meanwhile, the test-leader started the recording and carefully monitored the calibration procedure, which required approximately 2 s of visual fixation on the target. If the patient did not spontaneously look at the calibration target, or had their eyes closed, the assistant tried to attract the patient’s attention primarily through verbal encouragement or by patting an arm or knee. If necessary, the assistant changed their position to that of the patient’s habitual head and gaze direction. In order to reduce the risk of fatigue impacting on later response, no more than 3 attempts were performed before proceeding with the recording.

CRS-R visual function scale, test item “object recognition”

Testing was performed as per the CRS-R manual. Briefly, the patient is asked to look at 1 of 2 specified objects, the position of which is then alternated to confirm responses and the procedure is repeated. In this case (patient 2) calibration was successful. Two objects were presented, and the patient was asked to look at the ball; which, in this case, was located to their left (Fig. 1). An area of interest (AOI) was defined around each object and any visual fixation within this area was counted as a hit (Table III). Based on the distribution of hits in this example it was considered as an appropriate response.

In the next example (patient 1) the patient had a suspected alternating strabism; that is, the patient sometimes appeared to focus with the right eye and sometimes with the left eye. Apart from making clinical assessment more difficult this also meant that calibration was unsuccessful, and as such the visualization and metrics tools of Tobii Pro Lab could not be used. It also meant that no absolute eye positioning data could be obtained from the recording. Instead, relative changes in gaze direction were derived, based on changes in the gaze direction vector.

When observing the patient before commencing the tests, the gaze was habitually directed to the patient’s left corresponding to an approximate horizontal gaze direction vector value of 0.4 (Fig. 2). According to the coordinate system of the eye-tracking system, any increase in vector value corresponds to a leftward movement, while a decrease corresponds to a rightward movement.

The test was commenced by presenting 2 objects within the patient’s field of view; the ball to patient’s left and the cup to the right. The patient was asked to look at the tennis ball (Fig. 3).

A linear fit (right eye) resulted in an intercept 0.094, R-square 0.132 with a slope (2.847*10^-5) that was significantly different from zero. Based on visual inspection of the graph and the difference in slope it was considered plausible that this was an appropriate response; that is, an attempt to look towards the stimulus.

CRS-R visual function scale, test item “visual pursuit”

Visual pursuit was tested using a mirror held by the examiner, according to the CRS-R manual. In the first example (patient 2) the calibration was successful. This meant that the gaze point, left and right eye position averaged, could be plotted against the coordinate system of the scene camera. This coordinate system is depicted in pixels starting from the top left corner as seen from the patient’s view. Given the resolution of the scene camera this means that a gaze point at the extreme top left has the coordinates x = 0, y = 0 while a gaze point at the extreme down right has the coordinates x = 1,920, y = 1,080. The coordinates for a gaze point at the very centre are x = 960, y = 540. In order to reduce the complexity of the analysis, horizontal and vertical movements were plotted separately. As in previous cases, eye movement behaviour without any stimulus present (Fig. 4) was compared with eye movement behaviour when the actual test was performed (Fig. 5). On visual analysis the eye-tracking data was considered to show a correct response to the stimuli.

In the next example (patient 1) calibration was not successful. The first trial when the mirror was moved to the patient’s left failed to show any response. The second trial on the other hand, when the mirror was moved to the right, led to a response (Fig. 6).

CRS-R visual function scale, test item fixation

Fixation was tested according to the CRS-R manual. This proved to be the visual test item where it was most difficult to discern a response from eye-tracking data. After repeated trials visual analysis of eye-tracking data was considered to support occurrence of the desired response in one of the patients (patient 2) (Fig. 7).

CRS-R auditory function scale, test item “localization to sound” (assessed here as response is evaluated on the basis of head and/or eye movement)

Four trials were performed starting with a 5-s ringing of the bell on the patient’s right-hand side, as per the CRS-R manual. The expected response was the patient gazing or turning the head towards the direction of the sound. However, based on the eye movement recording and accelerometer registering head movements none of the trials resulted in what was judged to be such a response.

Clinical assessment of items vs eye movement analysis

A total of 24 trials over the 4 CRS-R subscale items were performed for each patient (Table IV) and data could be retrieved from a high percentage of trials (96–100%). Patient 3 was an exception with complete data for only 21% of the trials where the main issue was a gaze deviation. The agreement between clinical and eye-tracking assessment of responses was 69, 54, 80, and 67%, respectively, for different patients and was variable within subject and test item.

For patient 1 there was a disagreement between findings from CRS-R assessment and eye-tracking assessment on 7 occasions, whereas all of them were in favour for the eye-tracking protocol; that is, the eye-tracker found a positive response to the item where clinical assessment according to the CRS-R did not. For patient 2 there were a disagreement between the clinical and the eye-tracking protocol on 11 occasions, whereas 3 of them (27%) were in favour for the clinical protocol and 8 (63%) in favour for the eye-tracking protocol. For patient 3 there were only 5 complete trials. On one occasion there was a disagreement between the clinical and the eye-tracking protocol in favour of the eye-tracking protocol. For patient 4 there was a disagreement between the clinical and the eye-tracking protocol on 8 occasions, whereas 2 of them (25%) were in favour of the clinical assessment and 6 of them (75%) in favour of the eye-tracking assessment. In total, 27 disagreements were found between the 2 assessment methods and 22 of these (81%) were in favour of the eye-tracking protocol.

The objective of this study was to evaluate the feasibility of using wearable eye-tracking when assessing patients with PDOC using the CRS-R, focusing on technical challenges. Patient comfort, practical aspects, as well as technical requirements, were considered.

From a patient acceptability point of view the method seems feasible. No adverse reactions to wearing the eye-tracking frame were observed. However, with the current design of the frame, some adaptation and extra care was necessary. Our experience from this study is that ensuring the patient’s comfort, is key when performing the tests. Avoiding complete eye closure due to discomfort related to the eye-tracker equipment or the situation is an important prerequisite to assessment.

From a technical perspective, it is important to keep the eye-tracker frame in position for the sensors to capture eye movements correctly and also to prevent the frame from obscuring the field of view. The frame in itself limits the field of view, especially upwards, and care needs to be taken to prevent it from sliding down the nose. This is also key in order to minimize obstruction of the examiner’s view of the patient’s eyes. Some adaptations of the frame design, nose pads and ear pieces, may prove beneficial for these purposes.

Calibration of the eye-tracker was partially possible for some patients, despite severe neurological impairments. For patients where calibration was possible, the visualization and metric functions that are supplied with the system could be used. However, some limitations need to be considered. Even though some patients did keep their eyes properly aligned to allow calibration, observation sometimes suspected that alignment failed in some gaze directions. This may be a source of error, since the recorded gaze direction is based on averaging of the direction of each individual eye. When calibration was not successful only relative changes in gaze were available for analysis. Despite this, it was still possible to discern what appeared to be distinct changes that could be compared with the clinical assessment with the CRS-R.

For moving stimuli, dynamic mapping of eye movements with respect to the stimulus was not possible using the metrics and visualization functions supplied commercially. In this sample the size and direction of change in gaze direction during exposure to the stimulus, and not the absolute gaze position over time, was analysed. When this matched the movement of the stimulus visual tracking was considered to have occurred, although, theoretically, gaze direction could have been displaced from the stimulus; in short there remains some uncertainty as to whether the patient tracked the actual target. The intention had been to record eye movements during pre-test and then to compare them statistically with the changes in eye movements during the tests. This analysis was, however, challenging due to the short time-windows in which eye movements could be recorded. For example, the patient might close their eyes or look to the extreme left or right, resulting in interruption of the eye-tracking signal. We therefore had to rely largely on qualitative visual inspection of plotted data. This is less robust than a fully quantitative method would have been, but, given the known overall challenges in assessing these patients, we believe that this technology, with further development, may still add sensitivity to the CRS-R assessments. Future applications of this method would require technical adaptations to allow 1-eyed calibrations and dynamic mapping of eye movements vs stimulus. Preferably, this should be developed in cooperation with eye-tracking system providers, rather than local adaptations, to promote general availability.

In 3 out of 4 patients, eye movement data could be retrieved from a high percentage of trials. For one patient, data could be retrieved only from 21% of trials. The most likely reason for this was an extreme horizontal gaze deviation, which was suspected to be multifactorial, where, for example, impaired attentional processes probably contributed, rather than an isolated eye movement restriction. When data was sufficient for comparisons to be made there was strong agreement between the clinical CRS-R assessment and eye-tracking findings. For the other 3 patients, the agreement between assessments were 54%, 67%, and 69% respectively. The agreement was variable within patient as well as item. The test items “localization to sound” and “fixation” showed fairly strong agreement. However, these items were also the ones with the most unsuccessful responses, both in clinical CRS-R and eye-tracking assessment. In 81% of the trials where there was a disagreement between the clinical and the eye-tracker findings the eye-tracking data indicted a response, while the clinical CRS-R assessment did not, suggesting that eye-tracking could add sensitivity and, as such, minimize the risk that conscious responses are missed. However, this needs to be validated in a larger study. One aspect of CRS-R assessments that is seldom acknowledged is that it can sometimes be clinically difficult to determine whether the criteria for scoring an item are reached. Video-recording of responses (both clinical on assessment and with eye-tracking) also makes it possible to replay the session, if necessary, to support such judgements. Video-recording may also support training.

Previous research on eye movements in patients with PDOC has used stimuli presented on a computer screen (12–15), and reports that robust data were acquired in most cases. Presenting stimulus on a screen provides an advantage, in that it simplifies the mapping of eye movements with respect to the stimulus. It does, however, also require the patient to be in a specific body position to allow the sensors of the eye-tracker to capture the eye movements (12). It may also be more difficult for a clinician to observe the eye movements whilst recording the eye movements, due to the position of the computer screen. A wearable eye-tracker has greater potential in terms of practicality in both adjusting to the patient’s body position and allowing clinical observation during clinical CRS-R assessment.

To our knowledge, this study is the first described attempt to incorporate eye movement recording in CRS-R test paradigms using a wearable eye-tracker. This is a small study with the primary aim of evaluating the feasibility of incorporating eye-tracking in the CRS-R assessment. Further investigations with larger samples are needed to investigate its potential as a supporting tool when diagnosing patients with PDOC. One strength of this study is that the eye-tracking technology makes assessor-blinding possible when evaluating the effect. If consistent eye-movements can be detected earlier after brain injury with an adjusted wearable eye-tracker than with the use of the clinical assessment scale CRS-R, this could add sensitivity to the detection of subtle behavioural signs of consciousness, and provide an easy-to-use bedside instrument to aid the assessment of the most severely injured patients.

Based on the patient’s responses, and the possibility of discerning eye movement patterns, we conclude that the method appears feasible. Further study will help to clarify its full potential but the option of recording and replaying responses in these easily fatigued patients where the assessment sessions need to be brief is of great interest. The current hardware and software limitations can be overcome with some extra care and detailed analysis; however, significant developments will be required before introduction to routine clinical practice could be considered. Hardware adaptations and the calibration procedure appear to be important primary targets in this process.

The authors thank psychologists Iina Ervamaa Mekahal and Maria Pörn who performed the clinical CRS-R assessment, and Jonas Stenberg, PhD, who performed preliminary work leading to initiation of this project.

Ethics approval and consent to participate. The study was approved by the Regional Ethics Board, Stockholm. Since the patients cannot give consent themselves they were only included in the study if family relatives or the patient’s trustee did not oppose participation.

Funding statement. The study was funded by SLL Innovation (grant numbers 20170938 and 20190099) and Promobilia (grant no 18141).